NII-ISM, Japan at TRECVID 2007: High Level Feature Extraction

This paper reports our experiments on the concept detection task of TRECVID 2007. In these experiments, we have addressed two approaches which are selecting and fusing features and kernel-based learning method. As for the former one, we investigate the following issues: (i) which features are more appropriate for the concept detection task?, (ii) whether the fusion of features can help to improve the final detection performance? and (iii) how does the correlation between training and testing sets affect the final performance?. As for the latter one, a combination of global alignment (GA) kernel and penalized logistic regression machine (PLRM) is studied. The experimental results on TRECVID 2007 have shown that the former approach that fuses simple features such as color moments, local binary patterns and edge orientation histogram can achieve high performance. Furthermore, the correlation between the training and testing also plays an important role in generalization of concept detectors. 1 Feature-based Approach 1.1 Framework Overview In our framework as shown in Figure 1, features are extracted from the input keyframe image. In the training stage, we use these features to train SVM classifiers with RBF kernel. These SVM classifiers are then used to compute raw output scores for the test keyframe image in the testing stage. These output scores can be further combined by a certain fusion method for computing the final output score. In order to return K shots most relevant for one concept query, all normalized final output scores of shots are sorted in descending order and topK shots are returned. In the case of a shot consisting of several subshots, only the maximum score among subshots’ scores is used for that shot. 1.2 Feature Extraction We used three types of features including grid color moments, edge direction histogram (which are described in the baseline system [1]) and the local binary patterns. Fig. 1. The evaluation framework. The extracted features are normalized to zero mean and unit standard deviation and then stored for training and testing. Specifically, the normalized vector x = (x 1 , x 2 , ..., x N ) of an input raw vector x raw = (x 1 , x 2 , ..., x N ) is defined as follows: x i = (x i − μ) σ where x i and x raw i is the i-th element of the feature vectors x norm and x respectively, N is the number of dimensions. μ is the mean